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and a report of additional findings in the areas of project staff 
satisfaction, project staff perceptions of their needs, and targeted 
impacts of technical assistance. (Author/SW) 
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Assessment Strategies 
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ABSTRACT: The. Techiicat A&&l&tance. VaveZoprnznt Sy6tm (TAPS) con- 
ducted cm evaluation 0^ tioiee tethnical cu,6l&tance need^ cu>6eA&mervt 
iVuUegie^ duJunq 1979-80. The thAte iVuvtegie^ iwAe: (?) ON-SITE, 
--conducted by a tjuuned needi <ii>6t&6oA cut the pfLoject'i iite, (2) 
TllEPHOHE-- conducted by a tAained needs a^ie^6oA thA^ugh a &eAA.eJ> 
oi telephone conveAioUora , and (3) SELF-ADMHI I STERED-- conducted 
by the project's 6 ta^ a&ing pnaceduJie^ and matexLaU provided by 
TAPS. The 64 pfiogfuxm pcuitlcipatlng mAe /landomly assigned to one 
0^ the. thAee &Vuvtegle^. The n.(L6ultt> Indicated aZL th^ee 6tAategie^ 
adequately Identified technical a&6Utance need^ and weAe &<JiulaA 
on mo6t vahAjxble^. PeAceptionA of, the paAticipanti^ occasionally 
favofied on-tlte oveA telephone a&&es6mentt>. Comentt ^Kom paAtld- 
pant& suggested that uikUe the stAategles MeAe compoAable ($oa. 
Identifying technical assl&tanct needs, theAe. mqac some qualhxLtive 
chaAO-ctzAistlcs on which they diffeAjid. 
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BACKGROUND \ 

A new type of organization Is becoming prominent In government 
sponsored social programs. It Is the technical assistance (TA) agency. 
The general purpose of technical assistance agencies Is to provide 
assistance which will Improve the overall development, Implementation 
and/or evaluation of programs which have been funded to address a 
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particular social problem. Issue, or Interest. Technical assistance 
agencies have based their assistance on the knowledge and experience 
of others In the fields of: (1) utilization of knowledge, research, and 
Innovation , e.g., Havelock (1971 , 1973) ,, Rogers and Shoemaker (1971); 
(2) change , e.g. , Bennis, Benne and Chin (1969), Zaltman, Florlo and 
Sikorskl (1977); O) training , e.g., Craig (1976); (4) consultation and 
counseling , e.g.,, Blake and Mouton (1976), Lippitt and Lippitt (1978); 
(5) communication/resource linkage , e.g., Berlo (1960); and (6) 
organizational development , e.g., Schmuck, Runkel , Arends and Arends 
(1977),, Spencer and Cullen (1978). 

While the major approach or emphasis of particular TA agencies Is 
dependent upon their mission and the characteristics of their <;lients, 
they appear to have in common the inclusion of activities based on the 
concepts of: 

...(a) communication between a technical assistance 
provider and a client; (b) a linkage between needs 
and resources; (c) a ipechanism for the acquisition 
of new knowledge, competencies, and attitudes; 
(d) the provision of aid in the development of 
programs and organizations; and (e) a helping 
relationship between the technical assistance 
agency and client. (Suarez, 1980, pp. 18-19) 

The Technical Assistance Development System (TADS) is one such 
organization, funded under a contract from the Office of Special 
Education, U.S. Department of Education. TADS' purpose* is to provide 
comprehensive assistance to personnel implementing programs for pre- 
school handicapped children and their families. Both TADS and the 
programs^rved by TADS are components of the national Handicapped 
Children's Early Education Program (HCEEP). 

HCEEP funded Demonstration Projects, in the eastern United 
» 

States, are^one of TADS major client groups. The mission of these 
Demonstration Projects is to: (a) develop and implement model service 



delivery programs for young (birth to eight years) handicapped ch11d**en 
and their families, and ,(b) demonstrate these models to other agencies 
Interested In developing their own services for similar children and 

c 

families. 

TADS assists , these projects In managing their programs and 
accomplishing their goals more effectively through a broad range of 
systematic support arid consultative services. Access to expertise 
not available locally and to materials designed specifically for these 
projects Is available through TADS, TADS also provides opportunities 
to increase collaboration and Information sharing among the HCEEP 
projects. TiJeJlCEEP projects' participation in technical assistance 
is encourage^ but is voluntary. The TADS' services are available at 
no extra cost to the projects. ^ 



In order 
projects, TADS 
ment that face 



0 provide appropriate and effective assistance to HCEEP 
' personnel must know the needs or barriers to improve- 
the projects. The process of Identification of thes'e 
needs, needs assessment , is, therefore, a critical element in providing 
technical assistance* 

As Illustrated (Figure 1), needs assessment is one of the five 
major stages in the yearly cycle of technical assistance provided by 
TADS. It provides the Jink between the client's existing plans and the 
services provitjed by TADS. 

The primary purpose of the needs assessment is to Identify areas 
in which a clielijit's program might be enhanced by the receipt of out- 
side assistance. The needs assessment process is designed to accom- 
pllsh tllree major objectives: (a) to provide administrative technical 



Figure 1 ^_ ^ 



TADS MODEL OF TECHNICAL ASSISTANCE 




assistance to clients through a comprehensive program review and 
analysis; (b) to Identify technical assistance needs and a plan to 
address these needs; and (c) to establish a positive working relation- 
shl^ between the client and TADS. 

Technical assistance needs assessments are based on a set of seven 

assumptions developed by TADS: 

1 A systematic needs assessment process Is essential to 
effectively Identify Technical Assistance needs. 

2 A needs assessment must provide a comprehensive review of 
the client program to Identify current program status. 

. 3 Technical Assistance needs can best ber4dent1f1ed by a 
comprehensive review of the client's program. 

4. Needs are most accurately Identlfledjn' an Interactive 
process between the Technical Assistance provider and the 
client. ^ 

5. Roles, responsibilities, and expectations for the provision 
of Technical Assistance must be clarified during the needs 

assessment. , 

6 A needs assessment must establish and/or enhance a trusting 
relationship between the client and the Technical Assistance 
provider. 

7. A needs assessment must occur before extensive Individualized 
Technical Assistance 1$ provided. ^ 
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The activities or tasks which are considered necessary to accom- 
plish a needs assessment a«e divided into three sequential activity 
areas: preparrftion. Implementation, and follow-up'activities. Pre^ 
paratlon activities Include scheduling the needs assessment, preparing 
for the needs assessment, and establishing agendas for the assessment. 
Implementation activities include an overview and orientation to TADS' 
needs assessment, a comprehensive review of all program components, 
identification of technical assistance needs and determining their 
importance, developing a plan for technical assistance, and evaluating 
the needs assessment. Follow-up activities include a TADS' staff 
review of all needs assessment materials, the clarification of state- 
ments included in the materials if necessary, the development of an 
agreement specifying the needs and technical assistance to be 
delivered, and consensus from the client regarding the agreement. 

' During the first eight years of operation, TADS primarily employed 
an on-site needs assessment procedure which used a trained needs 
assessor (a TADS' staff person or TADS' consultant) who visited the 
project's site to conduct the needs assessment. This method of con- 
ducting needs assessment has proved to be highly successful in terms 
of client satisfaction and identification of technical assistance, 
needs. The costs of conducting this type of needs assessment, however, 
are high. With costs increasing, both TADS and the Office of Special 
Education were i-nterested ip, determining if less expensive needs 
assessment strategies could be used. As a result, the TADS' staff 
<lecided in 1979 to use several needs assessment strategies varying 
in cost and study the results of these assessments to determine if 
less costly strategies were feasible and effective. 

ERJC K 



aVERALL APPROACH 

% 

Evaluation Information was needed to make a decision, l .fe. , to 
aid in deciding which strategies might best and most economically be 
used to conduct a TADS' needs assessment. The Inquiry approach selected 
was that of Input Evaluation as defined by Stufflebeam (Stuff lebeam, 
1968; Stufflebeam, et a1., 1971) In his Context, Input, Process, C> 
Product (CIPP) evaluation model. Input evaluation, as described by 
Stufflebeam, Identifies and assesses alternative plans or strategies 
that might be choseyto achieve selected objectives or goals. Its 
purpose 15 to provide Information to make structuring decisions, ^ 
i.e., those that lead to the design or selection of procedures. 

The overall purpose of this evaluation study was to provide - 
information to decide if the selected strategies were compara.ble. 
If so, less expensive strategies could be used in conducting future V... 
TADS' needs assessments. An additional purpose was prompted by 
one of the few studies of needs assessment strategies (Clifton, 1969). 
This study suggested that different strategies might produce substantially 
different needs assessments. Therefore, another purpose was to provide 
evaluation information whicK would aid in determining if the selected 
strategies did produce the same needs assessment. 
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METHODS AND PRO C E D U R £ S 



DESIGN 



' StAotigy S(Lle.cXlon. After reviewing needs, assessment strategies 
employed by TADS in previous years and' the literature on needs assessment 
designs and characteristics of technical assistance efforts, the TADS' 
staff generated a brief description of 13 technical assistance needs 
assessment strategies. After reviewing each strategy, 10 were eliminated 
because they failed to provide'an adequate opportunity to meet one or 
more of the seven TADS' assumptions for needs assessments that were 
listed previously. The three remaining strategies, on-site , telephone, 
and self-administered needs assessments, were chosen for study. 

In order to determine if the strategies were comparable for 
conducting a TADS' needs assessment, it was necess^y to structure 
them so that they varied primarily in cost and not other areas. Sin 
the primary cost of technical assistance services are the travel 
expenses and fees for consultants, the sti-ategies^were designed to be 
identical in all aspects except needs assessor (or consultant) involve 
ment. 

More precisely, the strategies were designed as follows: 
1.' ON-SITE NEEDS ASSESSMENT (High Cost/High Needs Assessor 
Involvement) --All needs assessment implementation 
activities were conducted at the project site by the needs 
assessor and project staff under the leadership of the needs 
assessor. These implementation activities included the over- 
view and orientation to TADS and the needs assessment process,, 
the comprehensive review of the program, the identification 
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of project and technical assistance needs,, the determination 
of the priority of TA needs, and the developtnertt of a plan 
for technical assistance. Allspecified heeds assessment 
' materials were completed and sent to TADS by the needs 
assessor. The needs assessor, therefore, was involved in 
the entire needs assessment process. 
.2. TELEPHONE NEEDS ASSESSMENT (Md'derate Cost/Moderate Needs 
Assessor Involvement) — In this strategy the project 
staff conducted a programmatic review cind sent the results 
to the needs assessor. The needs assessor then reviewed 
the materials and completed the process with a project staff 
member* -through a series of three to five telephone conversa- 
tions. Then the needs assessor completed all materials and 
sent them to TADS. The needs assessor was; involved in some, 
but not all, needs assessment activities. 
3. SELF-ADMINISTERED ASSESSMENT (Low Cost/No Needs Assessor 
Involvement) —In this strategy the project staff 
completed all needs assessment activities following 
the procedures specified by TADS. One person on the TADS' 
staff was avail able^y phone during the assessment to answer 
questions and to clarify procefures. The project staff then 
sent all completed materials to TADS. There was no preplanned 
direct involvement of a TADS' trained needs assessor. 
EvaZuation Ve^lgn. Four major areas of comparison were selected 
on the basis of the information needs of TADS and its funding agency, 
the previous .experiences of TADS in conducting needs assessments, and 
the literature cited in the introduction section. The four comparison 
areas were: (1) integrity of the strategies, (2) perceptions of the 



participants, (3) level of time and effort, and (4) results of the 
technical assistance following needs assessment. These four areas 
were considered to be hierarchical in nature, i.e., the first area 
was most critical, the second was important and became a critical 
determinant if the strategies v^re comparable in the first areas, 

and so forth. - 

V Before considering other factors, comparisons in the first area, 
integrity of the strategies , wdre needed to establish that the three 
strategies were indeed variations of the same needs assessment^ This 
area was operationally defined as consisting of four variables. The 
first variable, "Criterion Met," was the extent which each strategy 
met. the established criteria Tor a TADS needs, assessment. Five criteria 
statements were drawn from the TADS' assumptions for a needs assessment. 
The five criteria statements were: 

1. The needs assessment provided a comprehensive review of 
the projecij. 

2. The needs of the project, those requiring and not re- 
quiring technical assistance,, were identified. 

3. A list of gilearly specified needs to be addressed by 
technical assistance was developed. 

4. Technical assistance activities for the needs described ' 
in item 3 above were identified. 

5. Roles and responsibiltlies for the project staff and 

TADS In planning and carrying out technical assistance were 
"clarified. 

The second variable, "similarity of needs," was defined as: (a) 
number of needs identified per project, and (b) type of needs identified 
in terms of their pr6graninatic content area and type of technical 
assistance needed. The third variable, "stability of needs," was the 
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extent to which the needs remained unchanged during the year. The final 
and fourth variable, "additibnal accomplishments," was the extent to 
which additional occurrences during the needs assessment were similar 
in number and type. 

If the integrity of the strategies was determined to be similar, 
then the next major concern was the area of perceptions of the partici- 
pants . Participants Include both the project staffs and needs assessors. 
Three distinct perceptions were investigated: (1 ) "project staff satis- 
' faction" with the needs assessment strategy In which they participated, 
(2) "appropriateness of the strategy" as perceived by both the project 
staffs and needs assessors, and (3) the "strengths and weaknesses" of 
the strategy as perceived by both the project staffs and needs assessors. 

The third area of comparison was level of time and effort . Three 
variables defined this area:' (1) "Calendar days for conducting" which 
was the number of days from the first scheduled day of the needs assess- 
ment to the day upon which the .needs assessment materials were received 
at the TADS' office; (2) "person hours for conducting" which included 
hours spent in preparation and Implementation by the project staffs, 
needs assessors, and TADS staff; and (3) "follow-up person hours for 
agreement development" which included only the TADS, staff time. (See 
page 5 for a description of the activities included in preparation, 
implementation, and follow-up for needs assessments.) 

The fourth and final comparison area was the results of technical 
assistance following needs assessments . Four variables defined this area: 
(1) "satisfaction with individual technical assistance services" which was 
a project staff rating for each technical assistance service described in the 
technical assistance agreement; (2) "progress in meeting needs" which ^ 

Er|c ^0 
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was the difference between the project' s, status In relation to 
identified needs prior to receiving Individualized TA and at the 
end of the year; (3) "ov.eralT Impacts of technical assistance" which 
were project ratings of both organizational and programmatic impacts 
of the technical assistance received during the year; and (4) "overall 
satisfaction with technical assistance" reported by project staffs at ' 
the end of the year. This comparison area was the furthest removed 
from the needs assessment and, therefore, the last area consid^ed. 
This was because the project staffs' perceptions of .the results of 
technical assistance were Influenced not only by needs assessments, but 
also by all the interactions with the TADS staff, TADS publications, and 
TADS' consultants during the entire year. It did seem impoKant, however, 
to know if responses to subsequent TA were in any way different for 
any of the strategies. 

In summary, the ^our major areas of comparison were defined by 
fourteen distinct variables. Figure 2 gives the complete design. 

PROCEDURES 

Sample. SeZe.ctA.on. Seventy-one projects composed the TADS' 
client groui^of demonstration projects during 1979-80. Two of these 
were excluded from the study because they were participating in a 
TADS case study which required d special needs assessment arrangement. 
Another project was excluded because it was different from all the other 
projects in the study, i.e., it did not provide services to children or 
parents directly. The total group which could be considered as subjects 
of the study were, therefore, sixty-eight demonstration projects. 
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Figure 2 



Input Evaluation Design to Determine the Comparibi lity of 
Three Needs Assessment Strategies 



Areas of Comparison 



Area 1 - INTEGRITY OF THE STRATEGIES 

1. Criteria Met 

2. Similarity of Needs 

3. Stability of Needs 

4. Additional Accomplishments 



Area 2 - PERCEPTIONS OF THE PARTICIPANTS. 

1. Project Staff Satisfaction 

2. Appropriateness of the Strategy 

3. Strengths and Weaknesses 



Area 3 - LEVEL OF TIME AND EFFORT 

1. Calendar Diiys for Conducting 

2. Person Hours for Conducting 

3. Follow-up Person Hours -for Agreement 
Development 



Area 4 - RESULTS OF TECHNICAL ASSISTANCE 
FOLLOWING NEEDS ASSESSMENT 

1. Satisfaction with Individual 
Technical Assistance Services 

2. Progress in Meeting Needs 

3. Overall Impacts of Technical 
Assistance 

4. Overall Satisfactipn with Technical 
Assistance , 
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The opinions of TADS' experienced staff and a review of the 
organizational change literature (Derr, 1976; Grelner, 1977; Spencer 
& Cullen, 1978) Indicate the results of different needs assessment 
strategies may be affected differentially by projects In different 
stages of development. Therefore, a stratified random sam'pling 
design, using year of funding as the stratifying variable, was used 
to assign projects to strategies (see Table 1). 



Table 1 

Stratified Random Assignment PRIOR TO ATTRITION 



Year of 
Funding 




)e of Needs Assessment 


On-site 
(N-22) 


Telephone 
(N-23) 


Sel f-Admlnlstered 
(N-23) 


First 
(N-18) 


6 


6 


6 


Second 
(N-21) 


7 


7 


7 


Third 
(N-29) 


9 


10 


10 



TOTAL N-68 



Since all Interactions and services from TADS are voluntary, 
three projects, which were In their third year of funding, chose to 
decline a needs assessment, and one project. In Its second year, 
chose to change to another strategy. All four projects were, 
therefore, eliminated from the study, leaving a total sample size 
. of 64. 



Table 2 

Assignment of Projects AFTER ATTRITION 





Tvoe of Needs Assessment 


Year of 
Funding 


On-site 
(N-21) 


Telephone 
(N-20) 


Sel f-Admlnlstered 
(N-23) 


First 
(N-18) 


6 


6 


6 


Second 
(N-20) 


7 


6« • 


7 


Third 
(N-26) 


8^ . 


8* 


10 



(N-26) I 8° I 
'Cells with Chanqe 
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Because attrition was low (less than 6%) and ther^e was little 
change In the proportions of projects In each cell, no changes were 
made In the original sampling plan or Implementation of th? study. 

Imptzmzntation o(J NeedA AA4W4mejtt SVuxtzgiz^. During September 
and October 1979, tf!e' directors/coordinators of the 68 (later 64) 
)nstrat1on prt^ects participating In the evaluation study were 
Informed whlcfwieeds assessment strategy had been assigned to their 
"and were asked which calendar dates they preferred to begin 
the, TADS needs assessment. Drawing from a cadre of approximately 
thirty previously trained needs assessors, TADS then assigned one 
needs assessor to each project having an on-site or telephone ni^i^ds 
assessment. Needs assessor assignment was made on^he basis of the 
same criteria for each client, i.e.. (1) the needs^^ssessor's . 
experience relative to the goals and objectives of the project, (2) 
^ TADS perception that the needs assessor and the project would be 
.compatible, and (3) aWilability. Since the basic procedures for 
conducting needs assessment strategies were similar to the 
procedures and materials used in the past, no additional direct 
training for needs assessors was deemed necessary for implementation 
of the study. 

Procedural manuals describing the steps in the needs assessment 
had been developed for each strategy for both project staff and needs 
assessors. They provided detailed instructions for cotiducting the 
assessments and were sent to^^he project staffs and needs assessors 
prior to the scheduled date of the assessment. The assessments were 
then implemented as scheduled. 
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Vata CotbQjcXAjon. The three groups responsible for providing 
information regarding the needs assessments were: (1) staff members 
of the demonstration projects, (2) TADS' needs assessors, and (3) TADS' 
staff members. Each demonstration project was responsible for report- 
ing the number of staff participating In the needs assessment and the . 
number of hours each was Involved. Also, the director/coordinator ^ 
had responsibility for completing a "Project Evaluation of Needs 
Assessment" form which gathered: (1) perceptions of the extent to 
which the 5 criteria for a TADS' needs assessment were met, 
(2) additional accomplishments of the needs assessment, (3) satisfaction 
with the needs assessment, (4) perception of the appropriateness of the 
strategy in which the project participated; and (5) perceptions of 
strengths and weaknesses of the strategy. The n^eds ^assessors also 
completed an evaluation form. The needs assessors' forms contained the 
same items as the project's, except the satisfaction items were omitted 

and an item requesting involvement time was added. The TADS' sta^ff 
maintained: (1) a time log for all of their activities associated 
with needs assessment; (2) a needs assessment monitoring log in which 
the dates of implementation and follow-up activities were recorded; 
and (3) a coordinator's monitoring notebook which contained copies of 
all agreements, notes about all transactions with the projects regarding 
delivery of services, and any changes made in the agreements.^ 

Data regarding the technical assistance received during the 
year was gathered on evaluation forms for each technical assistance 
service and an end-of-year survey. These forms and surveys, 
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completed by project staffs, provided Information regarding satis-' 
faction with the technical assistance, progress during the year in meeting 
identified technical assistance needs... and Impacts of technical 
assistance. 



r 



2. 
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RESULTS 



INTRODUCTION ^ 

""^s presented 1n the previous sections, the overaU purposes of 
this evaluation study were to decide If the three selected needs^ 
assessment t^trategles were comparable and If the strategies produced 
the san>e needs assissrnent. Sixty-four projects were classified by year 
of funding, and then randomly assigned to one of the three strategies. 
The four comparisons areas for the strategies were described and defined 
by 14 variables. The data associated with these variables were gathered 
: by the admlnistV-ation of sur'y&y^evaluatlon forms and m^ntenance of 

time and monitoring logs. The results, described 'in this section of 
the report, are based on the 4Tiformat1on collected by these forms and 
logs. The return and maintenance rates were very high: 90%, for all types 
of survey evaluation forms, and 100* for both the time and monitoring 
logs. Minor variations in sample sizes (or "n" values), from the total 
number of projects listed in Table 2 (page 13), are due to the varying 
return rates, and/or partial responses to a particular item on a 

returned survey form. 

The results of the study, described in this section, are in the 
order given in Figure 2 (page 12): integrity df the strategies, 
perceptions of the participants , level of tim e and effort, and results - 
of technical assistance following needs assessments . ^> 

In order to promote a focus on and an understanding of the results, 
descriptions of statistical tests and specific statistical analyses 
used are not Included in the text. Instead, they are referenced^in 
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the text and Included In Appendix A. In addition to these analyses, 
several addlt'lonal analyses We conducted In areas of 1i<^erest to the 
TADS^staff. These analyses, and the results associated with them, are 

V 

described In Appendix B. ^ 

t ■ . 

FINDINGS / " 

lYvtzQftUij oi the. SVicutagie^. The results of jthe analyses, of 
the four variables defining this comparison area. Indicated that all 
three strategies could be used to produce a TADS-speclfIc needs assess- 
ment. Responses from both the project staffs and needs assessors on 
the variable, "criteria met," Indicated that ^ill five criterion state- 
ments were rated highly (see Table 3). While the on-site strategy 
tended to receive slightly higher ratings, there was only one statis- 
tically significant different result. The project staffs participating' 
in the on-site strategy rated the Criterion statement, "roles/responsi- 
bilities," significantly higher than did the project staffs participating 
in the telephone strategy (see Tables 19 and 20 in Appendix A). However, 
the same result was not found for the ratings by the needs assessors. 

Analysis of the 168 needs, identified during the needs assess- 
mentis, indicated few differences among the three strategies on the 
second variable, "similarity of needs." There were no statistically 
significant differences (see Table 21 in Appendix A) in the average, 
number of needs identified per project by type of needs assessment 
(on-site, n»59, M=2.81; telephone, n-53, M=2.65; self, n^-Se, M«2.43). 
Statistical analyses of the type of needs identified, in terms of 
both prograirmatic content area ahd type Of technical assistance 
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Table 3 

Means and Standard Deviations of Responses 
Regarding the Extent to Which Criteria for 
the' Ne^ds Assessments Were Met 



w 1 1 1 1 O 


Project Ratings^ 


Needs Assessor 
Ratings 


) — 

on-site 
(n-21) 


telephone 
(n-19) 


self 
(n»22) 


on-site 
(n=21) 


telephone 
(n=19) 


1. Comprehensive Review M 

SO 


6.00 
.55. 


4.44 
1.25 


4.77 
.61 


4.57 
I. OS 


3.94 
1.16 


2. Project Needs lUentlfled M 

SO 

II III 


5.10 
.44 


4.68 
.94 


4.86 
.47 


4.71 

1.01 


4.47 
1. 01 


3. TA Needs Identified ^ 


5.19 
.40 


4.79 
.92 


4.71 
.56 


,4.81 
,93 


4.74 
.93 


4. TA Activities Identified M 

SO 


5.05 
.59 


4.68 
.94 


4.67 

.80 


4.57 
1.0S 


.4.74 
.93 


5. Roles/Responsibilities ^ 


4.95 
.59 


4.37 
.76 


4.95 
.64 


4.67 
1. 15 


4.84 
.60 



^Based on a 6 point scale where 1 ■ Not Met; 3 - Met Partially; 5 ■ Met 
Completely; and 6 ■ Exceeded Stated Criterion 
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needed, were not conducted due to the small numbers lii some categories. 
However, few- d1 fferences were discovered with most of these being In 
low Incidence areas (see Tables 4 and 5). 

Table 4 

Percentage of Needs Identified by Progranniatic Content Area 



Prograninatic Content Area 


Tvoe of Ne^ds Assessment. 


Total 
Needs 


On-site Telephone Self 

%V HIV 


High 


Demonstration/Dissemination 






Service for Children 


1 20% ZB% ZZ% 




Incidence 


Evaluation 


24X 2U 16« 


20% 


Low 
Incidence 


Services^for Parents 


B% 9% 16% 


m 


Admi ni strati on/Management 


B% B% 14% 




Staff Development 


12X 4X 5X ^ 7% 



'4, 
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Table 5 

Percentage of Needs Identified by Type of 
Technical Assistance Needed 



Type of Technical Assistance 




T^pe of Needs Assessment 


Total 




Heeded 




On-Slte Telephone 
(n-59) (n-53) 


Self 
(n«56) 


Moo/4 c 

(n-iee") 


High 
Incidence 


Infonmatlon 


m 


m 


m 




Revisions/Refinement 




20% 


23% 


23% 


22% 


Low* 
Incidence 


Skms/Competencles 


7% 


m 


9% 


9% 


Development 


lOX 


9% 


9% 


9% 




Planning ' 


lOX 


6% 


5% 


7% 




Decision-Making 


» 


i% 


2% 


7% 


4% 
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Th^ third variable, **stabnity of needs," was measured by the ' 
numbers of. changes in needs during the year. Overall, 20% of the 168 
identified needs changed. The on-site strategy had 14 (24%)*^ needs that 
changed, telephone had 12 {23%) that- chanqed, and self-assessments had 8 
(14%) yflianqes. These differences were not statistically significant 
(see Table 22 in Appendix A). ' 

The final variable of this comDarison area was measured by the 
number and type of "additional accomplishments," which were reported 
by the participants in the three strategies. While slightly more 
additional accomplishments were reported by the on-site participants 
(see Table 6), the differencies were not statistically significant 
(see Table 23 1n Appendix A). A review of the types of additional 
Accomplishments, described by particioants, revealed that the program 
review and planning were the most frequent additional accomol ishments 
for all strategies (see Table 6). Other common accomol ishments related 
to increased staff awareness, understanding of their project, and 
increased staff morale. Strategies involving needs assessors oroduced 



unique accomp.lishments, e.g. , the orovision to the stSff of new informa- 
tion in on-site and telephone assessments, and the opportunity for 
staffs to share, their successes during on-site assessments. 

■ ■ ■ Tfcble'€ ■ . ' 



Percentage of Projects and Needs Assessors Indicating Additional 
Accomplishments AND Frequency of Types of Additi onal Accompl i shroents 





Additional Accomplishments 


Project §ta++>' 
ResDonses 


Needs Asses- - 
sor^ Responses 


a: 

UJ 

00 






ON- 

C TTr 

SlTt 

n=19 


TELE- 

DunNC 

rnUKt 

n=14 


n=21 


ON- - 

^TTF 

O 1 1 L 

n=20 


TELE- 
PHONE 
n=16 


z> 
as- 




Number of Projects 
Percentage of Projects 


68% 


8 

57% 


12 

57% 


13 
'65%' 


9 

~ v56% 




1 


Program Review/Pl anni ng 


. 4 




8 


3 


2 




o' 
u 


New Information 


4 


1 




3 


4 




o 

<£ 
(O 


Undei^tanding Role as , 
HCEEP Project 


2 


3 


2 


1 


1- 




c 
o 

••- l/> 


Increased Staff Morale 


1 


1 


I 


2 




TYPE 


4-> 4-> 
•1- C 

TD g 
< g 


Opportunity to Share 
^ucesse^ 






' I ; 








o 


Increased Staff Aware- 
ness bf Roles 


2 




2 








quenc 
shmer 


Increased Awareness of 
TADS Role , 




1 




2 ' 


T 




u^^ 


Other f 








1 


1 




TOTALS ^ 


13 




13* 


16* 





These columns total more than the number of projects or 
needs assessors indicating an additional accomplishment 
because more than one accomplishment comment was listed. 



?(tAt<LptLoni> oi th& PoAUcipantA. While some similarities were 
found in this comparison area, differences did emerge, regarding the 
perceptions of the participants about the on-site and telephone strate- 
gies. The first variable, "project staff satisfaction," was measured 



by project staffs "ratings of foyr satisfaction items. While the self- 
assessment had slightly higher ratings than the other strategies (see 
Table 7^, none of the differences in the ratings of the four items 
we|e statistically sign-^ficant (see Table^4 in Appendix A). 

Table 7 ' , ' ' 

Means and Stalidard Deviations of Project Staff i 
Ratings of Sajtisf action with Needs Assessments 1 



Satisfaction Itemi 




>ro.iect Staffs' Ratinas^ 


3n-S1te 
(n=21) 


Telephone 
(n=14) 


Self 
(n=2n 


T. Expectations Met 


H 

. 5P 


1.00 ; 


4.53 
.77 


4.76 
.54 


2. Usefulness 


M 

^ SO 


4.81 
.93 


4.37 
. 9i 


4.91 
.53 


3. Quality 


'M 
SO 


4.81 
.6S 


4.24 
1.09 


4.95 


4. Overall Satisfact 


■ion 1 


4.61 
.87 


4.29 

' 1. 26, 


4.82 



^Based on a 6 point sckle where 1 = Unsatisfactory; 3 = 
Average; 5 = Excellent; and 6 » Exceptional. 



The second variable, "appropriateness of the strateciy," was 
measured by the project staffs and needs assessors responses to the 
questions: "If given a cUice, which type of needs assessment do you 
believe would have been mist effective for the oroject this year'?;" 
and "What are the reasons for this choice?" The participants in the 
on-site strategy tended to choose that strategy as most appropriate 
more often than participants in the other two strategies (see Table 8) 
However, only the differerices between the project staffs and needs 
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assessors participating iff the on-site and telephone strategies were 
statistically significant (see Table 25 in Appendix A). 

Table 8 

Frequency and Percentage of Project Staffs and Needs Assessors 
Indicating that the StrategyHn Which They Participated Was Most 
Appropriate. 



Appropriateness 


Pmiect Staffs 


Needs ^ 


ssessors 


On-site 
(n»2n 


Telephone 
(n-20) 


Self 
(n=23) 


On-Site 
(n-21) 


Telephone 
(n=20) 


Frequency of Agreement 




7 


13 


16 


4 


Percentage of Agreement 


81% 


35% 


57% 


76% 


20% 



\ The needs assessment participants provided reasons to support the 

choices they made. If they chose the same type of strategy in which 
they participated, then their reasons were perceived to provide support 
for the appropriateness of that strategy. Conversely, if they chose 
a different strategy, the reasons were perceived to support the inappro-. 
priateness of the'strategy in which they particpated. Tables 9 and 10 
contain the reasons that were listed by two or more participants. All 
of the reasons classified in the "other" category are unique responses. 

The participants in the on-site strategy listed different reasons 
for both appropriateness and inappropriateness than either the telephone 
or self-administered strategy participants (Table 9 comnared to Table 10) 
The reasons given for support of the on-site strategy were all 
associated with the presence of a needs assessor (see Table 9). The 
only cormonly mentioned reason for the on-site strategy being inappro- 
priate was when the project staffs believed that they knew their own 
needs . . 
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Table 9 



Reasons for Appropriateness or Inappropr^ateness Given by 
Qn-Slte Needs Assessment Participants 







Frequency of Reasons® 


Reasons 


Project 

Staffs 

(n=20) 


Needs 
Assessors 
(n=20) 


1 ^ Appropriate 


1. Opportunity for face to face 
Interaction with needs assessor 


2 


5 


2. External/objective view of the 
program can be obtained 


4 




0. rrugraifi Vr'^^c^urcS ^nu indtcriais/ 

can be observed as It operates • 
as a source of Information for 
determlnlnc) needs 


4 




4. Needs assessor can gain better 
understanding of program and 
become acquainted with staff 


2 





5. Project and/or staff members are 
new 


— 


6 


6. TADS process of needs assessment 
and technical assistance available 
can be explained 


— 


4 


7. Technical assistance can be 
provided during the heeds 
assessment 




2 


8. Positive feedback and/or support 
can be given to staff 




2 


9. Other 


7 


6 


Inappropriate 


1 . Project staff knows and can 
. Identify own needs 


, 2 




2. Other 


3 


4 



^The total number of reasons may be more than the number of 
projects or needs assessors because more than one response 
was given. 
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Table 10 

Reasons for Appropriateness or Inapproprlateness Given 
by Telephone and Self-Administered Needs Assessment Participants 



Reasons 



FreQuencv of Reasons 



Telephone 



(0 



e 

10 

c 



3. Project staff knows and can 
identify Its own needs. 



reject 
Staffs 



Project staff knows TADS needs 
assessment process and can 
implement It. 



Process is efficient in terms 
of time, energy, and/or cost. 



4. Other 



1. On-site assessor can aain a 
better understanding df the 
program and become acquainted 
with project staff. 



2'. Project staff knows TADS need 
assessment process and can 
Implement it. 



Technical assistance can be 
provided during an on-site 
needs assessment. 



TADS process of needs assessmen 
and technical assistance 
available can be best explained 
on-site. ^ 



S. Other 



Needs 
Assessors 



Project 
Staffs 
(n"18l 



10 



"The total number of reasons may be more than the number of projects 
or needs assessors because more than one response was given. 
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There was some overlap In the reasons given by the participants 
in support of the telephone and self-administered strategies (see 
Table 10). These reasons focused on the efficiency of the strategies 
and the abi'MJ^y of the staffs to conduct major portions of the 
assessments. The needs assessors participating in the telephone strategy 
were the only participants in either the self or telephone strategies 
which gave common reasons for the Inappropriateness of the strategy^ 
These reasons were associated with the project 5itaffs' inability to 
conduct the needs assessment, and the loss of benefits derived from 
the needs assessor not being on*-site. 

The third and final veiriable of this comparison area was "strengths 
and weaknesses." Participants in all three strategies indicated 
there to be strengths and weaknesses in the strategy in which they 
participated (see Table 11). Most participants found there to'be 
strengths in all three strategies. The differences in perceptions 
of strengths* by the participants, among the three strategies were, 
not statistically significant (see Table 26 in Appendix A). 



Tablell 

"^equency and Percentage of Participants Indicating That There Were 
Strengths and/or Weaknesses of the Strategies 



Type of Comments 


Needs Assessment Strateav 


On-site 


Teleo 


hone 


Self 


Project 
Staff 


Needs 
Assessors 


Project 
Staff 


Needs 
Assessors 


Project 
Staff 


STRENGTHS 

.-Frequency 


(n-ig) 


(n-18) 


:n-15} 


ln-18) 


(n-21) 


14 


14 


12 


n 


13 


Percentaae 




7M 






62% 


Percentage 


(n-18) 




(n-15) 


(n-l^J 


(n-J6J 


— 5— - 

22« 


2-— . 

35^ 


6755 


IJ 

IW" 


8 

-"WC" 



o 
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Strengths of all the stV-ategles were described by the participants 
to be benefits to the staff and the comprehensive review (see 
Table 12). More strengths were listed for onTslte assessments and were 
associated with the people Involved, I.e. , benefits to staff, the needs 
assessor, preparation of the staff, etc, Strengths of the telephone 
assessment Included both the procedures and benefits to the staff. Most 
of the strengths listed for self-administered assessments were associated 
with the TADS heeds assessment process. 

Participants in the telephone strategy Indicated more weaknesses 
than either the self or on-site strategy participants. However, only the 
responses of the proj^t staffs participating in the telephone strategy 
were statistically significantly different from the project staffs in 
% on-site strategy (see Table 27 in Apoendix A). Most of the weaknesses 
listed by participants in all three strategies were associated with 
the specific procedures and/or rnaterials used to conduct the assessments 
and were aimed at their detail and seemingly rigid structure. In addition, 
a weakness listed for on-site assessments suggested a lack of flexibility 
or adaptation of the process to the unique characteristics or needs at 
the project site. The notable weaknesses of the telephone assessments 
had to do with the amount of time required to complete the needs assessment. 

Leve£ o(J T-une and EUo^vt. Analysis of the first variable associated 
with this comparison area, "calendar days for conducting," indicated that 
the telephone strategy took a greater number of days to compute, self- 
assessment strategy next, and on-site strategy took the fewest number of days 
(see Table 13). All three comparisons of the number of calendar days for 
conducting the needs assessments wer^ statistically significant (see 
Tables 28 and 24 in Appendix A). 
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Table 12 



Strengths and Weaknesses Conments About the, Strategies 
As Stated by the Participants 





Freauencv of Comments* 


Uv/lllliCll L 9 


On-site 


Tele 


}hone 


Self 




Project 
Staffs 


Needs 
Assessors 


Project 
Staffs 


Needs 
Assessors 


Project 
Staffs 


o 1 i\Lnu 1 no ^ 


(n«14) 


(n=13) 


(n-11) 


(n-9) 


(n=13) 


1. Benefits to staff 
(e.g.t better under- 
standing, increased 
morale, etc.)' 


3 


3 


2 




3 


review 


c 
3 




3 


— 


3 


3. Needs Assessment 

MKl UCi 1 a 1 9 • 




3 


2 


4 




4. Structure Approach 








2 


2 


5. Identification of 

weaknesses, direc- 
tion provided for 
future activities 


2 


— 


— 


— 


4 


6. Establishment of 
nncitivp rpldtion- 
Ship with TADS 


2 










7. Preparation of 
staff prior to 
needs assessment 




3 


— - ■ 


2 


—- .. , 


8. Needs assessor 
(e.g. , expertise, 
flexibility, 
familiarity with 
project, etc.) 


4 


— 








9. Other 




6 






i 






(n-5) 


(n-n) 


in-3j 


ln-8j 


1 . Needs assessment 

too detailed, 
lenqtfiVt etc.) 


2 




4 


— 


4. 


process 




3 


6 

* 


4 




3. unique nature of 
the project was not 
addressed 


2 










4. Rigidity of pro- 
cedure 




. 3 








5. Failure to obtain 
outside, objective 
view of project 










2 


6. Other 


1 


1 ■" 


4 , 


b 


3 



The total number of corrments may be more than the number of projects or 
needs assessors because more than c^e conment was made. 
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Table 13 

Calendar Day for Conducting the Needs Assessment 







On-site 
(n-21) 


Telephone 
(n-20) 


Self 
(n-23) 


Number of Days 


Median 
Range 


8.4 
2-50 


31.5 
3-121 


13.3 
6-«5 



The second variable, "person hours for conducting," and the third 
variable, "follow-up person hours for agreement development*" tended 
to Indicate that on-site needs assessments required the most Invest- 
ment of person hours (see Table 14). However, the only statistically 
significant difference was the number of person hours for conducting 
the on-site strategy when compared to the telephone strategy^. On the 
average, the on-site strategy took significantly more person hours to 
conduct than the telephone strategy (see Tables 30 and 31 In Appendix A). 

Table 14 

Person Hours for Conducting 
(Includes- Project Staffs, Needs Assessors, and 
TADS staff) AND Persons Hours for Follow-up 
Agreement Development (only TADS Staff) 



Person Hours 


On-Slte 
(n-21) 


Telephone 
(n-20) 


'Self 
(n-23) 


Conducting ^ 
SO 


58.56 
Z0.30 


37.08 
19.40 


48.65 
30.68 


M 

Follow-up ^ 


3.56 
2.63 


2.43 
1.6S 


2.99 
2.76 



ReAottA o(( Te.<ihni(ULt Ki6i6tan&e. FottoMing NeedA Aisie^imtnt. 
The results of the analyses of the four variables comprising this 
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area Indicated there to be no significant differences amfng the 
strategies (see Tables 33 to 36 In Appendix A). 

When project staff rated their overall satisfaction with each 
of the technical assistance services provided as a t-esult of the 
needs assessments, highest ratings were given- by staffs who had 
participated In on-site and telephone assessments (see Table 15).' 

Table 15 



Satisfaction With Individual Technical Assistance Services 





Prolect Staffs Mean Ratinas^ 


Satisfaction Item 


On-site 


Telephone 


Self 




(n-S?)** 


(n-49)'* 


(n-50)'' 


Your overall satisfaction ^ 
with this service £E 


5.23 

.71 


5.22 
.69 


4.98 

.89 



^Based on a 6 point scale where 1 - unsatisfactory, 3 - average, 
5 ■ excellent, and 6 - exceptional. 

''These "n" values equal the number of Individual evaluation 
forms returned after each Technical Assistance service. 



Those staffs participating In on-site and self-administered 
assessments Indicated the greatest progress In meeting their Identified 
needs (see Table 16). They believed, however, that they were hot 
as far along In meeting their^ needs before technical assistance as 
those staffs who participated In telephone assessments. The status of 
all projects In relation to Identified needs at the end of the year 
was similar for all three types of needs assessments, accounting In 
part for the smaller degree of progress for those participating In 
telephone assessments. ^j^^^xS 
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Table 16 
Progress In Meeting 



Status of Need 


p».niort <;t/iff5 Mean Ratlnas^ 


On-site 
(n-18) 


Telephone 
(n-17) 


Self 
(n=18y 


Status Before ^ 
Technical Assistance '§£ 


2.38 

1.07 


2.63 
0.98 


2.3a; 

i.ii 


Sthtus at End M 
of Year ££ 


3.94 
1.00 


3.82 
0.97 


3.86 
1.08 ^ 


Amount of Change In ^ 
Year ££ 


1.56 
1.11 


1.19 
0.9) 


1.47 
l.OA 



^Based on a 5 point scale where 1 - not begun to plan work on 
this need, 3 ■ began to Implement some of the activities for 
this need, and 5 ■ completed desired work on this need; 



The highest degree of impact of technical assistance was reported . 
by project staffs participating in on-site assessments (see Table 17). ' 
For organizational Impacts, this result was followed by the Impacts « 
reported by staffs DarticlDating in teleohone assessments. In relation 
to progratmiatic aspects, the on-site results were followed by a 
slightly higher degree of Impact for those who participated in self- 
administered assessments. 

When staffs rated their overall satisfaction with all of the 
technical assistance they received during the year, i.e., needs 
assessments, individual TA services, workshops and publications, 
highest ratings were given by those who participated in on-site assess- 
ments followed by those conducte;!^ telephone and then those which 
were self-administered (see Table 18). 



Needs 



I 

Table 17 



Organizational and Progranmatic Impacts of All 
Technical Assistance Services 



Impact Area 


Prolect Staffs Ratlrias of Impacts" 


On-site 
(n-16) 


Telephone 
(n-17) 


Self 
(n-17) 


M 

Organizational - 

■ SD 


36.25 

16. U 


32.1)8 


27.65 

16.98 


Progranmatic ^ 


30.06 

14.97 


24.65 

13.58 


25.35 

19.31 



"Based on a 7 point scale (-3 to +3) where -3 « significant 
negative Impact, 0 ■ no Impact, and +3 ■ significant positive 
Impact. The Impact ratings are the mean of the total oer oroject 
scores for all Items for each area (24 Items for the organi- 
zational area and 26 Items for the programmatic area). 



Table 18 . 



Overall Satisfaction With All 
Techn:1cal Assistance Services 



Satisfaction Item 


Project Staffs- Ratings" 


On-site 
(n-18) 


Telephone 
(n-16) 


Self 
(n-17) 


— 1 , 

Your satisfaction with » 
this year's TADS ^ 
technical Assistance — 


2.56 

.70 


2.44 
.63 


2.41 

.80 



Based on a 7 point scale (-3 to +3) whfere --3 ■ 
extremely dissatisfied, and +3 « extremely satisfied. 



SUMMARY OF FINDINGS 

The first comparison area, "Integrity of the strategies," 
established that all three strategies did produce a TADS-speclfIc 
needs assessment. The second area, "perceptions of the participants," 
occasionally favored the on-site strategy over the telephone strategy, 
while the perceptions about the self-administered strategy did not 
substantially differ from either the on-site or telephone strategies. 
Analysis of the variables associated with the third area, "level 
of time and effort," indicated some significant differences. The 
telephone strategy took the longest number of calendar|days to complete, 
self-assessment stratf^gy next, and the on-site too,k the fewest calendar 
days to complete. Also, the on-site strategy took significantly more . 
person hours to conduct than did the telephone strategy. ^The final 
comparison areV, "results of technical assistance following needs ^ 
assessment," revealed no statistically significant differences among 
the strategies. 

Figure 3 provides a suirenary of the statistically significant findings 
related to the 14 variables which defined the fbur ma.ior areas of 
comparison. 

\ 
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Figure 3 



S«™ry of Findings to Detenilne the Co»pir1bnity of Three Needs Assessment Strategies 



Areas of Coiraparison 



Area 1 - INTEGRITY OF THE STRATEGIES 
1. Criteria Met: 



a. Criteria 1-4 

1 

b. Criteria 5 



More than Tele- 
phone (PS)^ 



2. > Similarity of Netds 



3. Stability of Heeds 



Statistical FIfMJIngs 



On-site 



N 0 



N 0 



N 0 



Telephone 



0 1 F F E R E N 



Less Than On- 
Slte (PS)^ 



Self 



No OlfferencV 



D I F F E R E N 



C E 



0 I F F E R E N 



C E 



Additional Acconpllshnents 

AreiL-2 - PERCEPTIONS OF THE PARTICIPANTS 
1. Project Staff Satisfaction 



N 0 



N 0 



D I F F E R E N 



D I F F E R E N 



C E 



C.E 



2. Appropriateness of the Strategy 



More Approprlati 
than Telephone 

(PS. NA)^'^ 



Less Appropriate 
than On-site 

(PS. NA)^'^ 



No pifference 



^ ^Strengths and Weaknesses 



Fewer Meaknesie! 
than Telephone 



More Weaknesses 

than On-site 



No Difference 



Area 3 - LEVEL OF jm AND EFFORT 
1. Calendar Days for Conducting 



Less than Tele 
phone' and Self 



More than On- 
Site and Self 



Less than Tele- 
phone; More than 
On-site 



2. Person Hours for Conducting 



More than Tele 
phone 



Less then On- 
Sltte 



No Difference 



y*3. Follow-up Person Hours for Agreement 
Development , 



.-NO 



D IFFEREN 



C E 



Area 4 - RESULTS OF TECHNICAL ASSISTANCE 
FOLLOWING NEEDS ASSESSMENT 

' 1. Satisfaction with Individual 
Technical Assistance Services 



N 0 



D I F F E R E N 



2. Progress In Meeting Heeds 



N 0 



D I.F F E R E N 



C E 



C f 



3. Overall Iinp^cts of Technical 
Assistance 



N 0 



D I F F £ R E N 



C E 



4. Overall Satisfaction with Technical 
Assistance 



N 0 



D I F F E R E N 



C E 



Vs • Project Staff 
^HA • Heeds Assessors 
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DISCUSSION 

» t ' ■ ' 

A great deal information has been presented about each of the 
three strategies and t\}eir comparability. This final section of the 
report will provide a dikussion of the findings and an explanation 
and description of the decisions which were made relative to them. 

DISCUSSION OF THE FINDINGS . " ' , 

T.Yiti.Qfvuty 0|{ thz' StAjOitQ.Ql<u . Unlike the results obtained by 
Clifton (1969), the results of this study indicate that on-site, 
telephone, and self-administered needs assessments can be used to 
conduct a specific, in this case, TADS needs assessment. All three ^ 
strategies niet the TADS criteria for needs assessments, produced similar 
numbers and types of needs* had similar numbers of changes in needs, and 
produced additional accomolishments. While thes number of addi- 
tional accomplishments was similar, the variety in the types of addi- 
tional accomplishment comments indicates that sjipplemental benefits do 
occur when, the level of needs assessor involvement increases. 

' P^captLoM, of, th(L PoJuUxUpanti . SeveraMindings suggest that 
telephone assessments were less favorably regarded by their particpants 
than on-si^e^sessments. ^^Teplione* assTssi^^^ 
perceived as the most appropriate strategy and ^mo.re weaknesses were 
identified for this strategy. In addition to these findings, comments 
made by the participants provide further information about the unique 
characteristics of each strategy. Favorable perceptions of the on-site 
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strategy reflected the benefits of the needs assessor's high involvement, 
which included the direct benefits to the staff of increased knowledge, 
understanding, and morale; artd the external/objective view of the 



program provided by the needs 
telephone andaiflself-assessment 



assessor. Favorable perceptions of the 
strategies focused on the efficiency of 
the process and the capabilitlies of the project staffs, e.g., their 
knowledge of the needs assessment process and understanding of their 
)wn needs. 

These findings suggest that if the telephone needs assessment 
^rategy is going to be used in the future, it needs some modifications 
i:o improve the areas which cniated negative participant opinions in this 



study. More importantly, the 



above findings provide some insight Into 



characteristics of the client^ which may be best matched with the level 
cf needs assessor involvement] Tliose projects needing a more in-depth 
understanding of jtheir program and the role technical assistance may 
play in their development, and/or needing direct support and assistance 
for the staff, might best be served by a strategy which has high needs 
alssessor involvement.^ ^Needi lalf^^^s^ment strategies with no needs 

best matched with projects whose staffs 
eeds assessment process sufficiently well 
t(|) conduct an effective assessjnent. 

! Lzij^t oijimz and E^^ont. Differences were again evident when time and effort 
variables were considered. By design the on-site assessments were 
more expensive in terms of costs to the techntcat assistance agency, TADS, 
followed by telephone and then self-administered assessments. The, 
results of the study indicated that on-site- assessments also required 



ssessor involvement might be 



know their own needs and the n 
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the greatest amount of effort in tenns of person hours expended. On- 
site assessments, therefore, are the most consuming of personnel and 
financial resources. In terms of calendar time, however, they are the 
most efficient. Self -administered assessments are the least costly, 
take moderate personal effort to complete when compared to on-sites and 
somewhat more calendar time to complete. 

Since one of the primary reasons prompting this evaluation study 
was to ascertain if less costly needs assessments could be employed, 
the results In this comparison area could suggest t*iat telephone and 
self-administered strategies should be used. However, before implementing 
such a decision, consideration must be given to the factors of total 
Implementation time, amount of personal involvement, and the matching 
process discussed in the previous section. 

Increased calendar days to conduct needs assessments require more 
time from the technical assistance agency's staff to monitor the process 
and more importantly, decrease the time that can, be allotted to the sub- 
sequent technical assistance services to meet the identified needs. For 
these reasons » caution should be exercised in choosing a strategy. In 
this case telephone assessments, Whi.ch requires substantial lengths of 

time to complete. 

A belief that guides the assistance provided by TADS is that 
personal involvement in technical assistance is necessary to br1n|^ 
about change. This personal involvement is particularly true for 
needs assessments since the needs identified during the assessments 
will often require a coiimitment to change by the entire project staff. 
The decreased 'person hours for conducting the telephone needs assessments 
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provide some evidence for less personal' involvement and, therefore, 
should not necessarily be considered a favorable outcome. 

Rz6iiU6 0(J Te,ahnical fii66l6tancLe, FoJUouxing Weedi A^^w^ment^. While 
the results reported in this comparison area tended to favor on-site 
assessments, none of the differences was statistically significant. 
Therefore, it may be concluded that the type of needs assessment strategy 
employed (on-site, telephone, or self) does not appear to affect the sub- 
sequent satisfaction with individual services, progress in meeting needs, 
impacts of technical assistance or satisfaction with all technical 
assistance. 

THE DECISIONS MADE ' 

Two decisions were made by the TADS staff based on the results of 
this evaluation study. The first decision was to use both the on-sitp 
and self -administered strategies to conduct future TADS needs assessments^ 
In general, the results indicated that the telephone strategy was com- 
parable and it was not eliminated for this reason. Rather, the telephone 
strategy was eliminated becausei^f the more negative opinions about it, 
and the logistical problems experienced (e.g., more calendar days to 
complete). This made the telephone strategy less desirable than the less 

expensive and mofe favorably perceived self -administered strategy. 
The second decision was to conduct on-site assessments with all 

first year projects and those second and third year projects who, in 

their own opinions and those of TADS, would most benefit from high needs 

assessor involvement. The remainder of the projects would conduct self** 

administered needs assessments. There were several reasons for making 
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this decision. Since projects In their first year' of funding win have 
had no previous experience with T^DS services and needs asses^nts 
procedures, it was believed that they would benefit more from the on- 
site strategy. The benefit is that a needs assessor, on-site, could 
explain the role technical assistance can play in^the development of 
their project and provide an in-depth experience in and explanations 
Of the TADS needs assessment procedures. » An on-site needs assessment 
during the first year would, therefore, provide a better opportunity 
for the project to: (1) effectively understand and use TADS services 
and (2) participate in the less costly self-assessment strategy in 
subsequent years. The on-site strategy would only be conducted 
with second and third year projects when the staff believed 
there to be a need for an assessor to be on-site and/or when major 
changes in the project occur which would affect the staff's capability 
' to conduct the assessment, e.g., key leadership ch&nges, etc. Self- 
administered assessments, therefore, .would be cbnducted by the leader- 
ship and staffs of second and third year projects who know the TADS 
needs assessment process and can, in their opinion and that of the 
TADS staff, effectively Identify their own needs and benefit from a 
self assessment of their^ program. 
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This appendix contains the statistical analyses described In the text. 
As stated previously, the purpose of the study was to determine If the three 
strategies were comparable, I.e., that there were not differences among th6 
strategies In relation to the variables that were assessed. Statistical 
, technloues were chosen which would determine If there was a statistical 
dlffet^ence'^npng the strategies. 

Choice of technloues was based on the level and tyoe of the data that 
were collected. One-way analysis of variance was used when the data were 
of at least an "1nt*>rvaT' level, e.g., LI kert-tyoe Items, number of person 
hours, number of needs, etc. Chl-Square analysis for Independent samples 
was used >hen the results were at a "nominal" level and produced freauencles 
of, for examole, "yes" and "no" responses, e.g., additional accomplishments, 
appropriateness, strengths and weaknesses, etc. In one Instance, number of 
calendar days, the data were at an "Interval" level but were so scattered^ 
that a r^e^n;^or average, would have been an inaporopri ate measure of central 
tendency. In this case, medians and ranges were calculated and the appropriate 
statistical test for this type of data, the Kruskal -Wall Is One Way Analysis of 
Variance Test (Siege! , 1956), was conducted. 

When a statistically signlf Icant'dl fference among the three strategies 
was found, additional comparison tests were conducted to determine where the 
differences occurred. When an analysis of variance was conducted and found 
to be significant, the foUow-uD test used was the Scheff^ MuUlole Comparison 

4'j 
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Test. When a Chi-square analysis was significant, additional chi-squares 
were computed comparinq two strategies at a time. For the Krus^^l -Wall is, 
. the follow-up test used w9s the Median test. * ^•^=i»*^ 

The level pf significance chosen to indicate that there was a signifi- 
cant difference for all analyses was set at ^.OS. 

On the following pages are tables presenting the results of the statis- 
tlcal analyses In the order that the data are presented In the text and where 
available, i.e., those conducted^ by computer, the specific £ level. Asterisks 
denote those differences th«it were significant. y 
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Table 19 



Analyses of Variance: Extent to which Criteria Were Met 



Source of Variation ^ 



df 



MS 



AFF RATINfiS 




PROJECT 

7 

Comprehensive Review 
Type of Needs Assessment 2 1.44 2.12 .1303 

Error 52 .68 

Project Needs Identified 
Type of Needs Assessment 2 \<81 1.90 .1593 

Error 53 .43 

Technical Assistance Needs Identified 
Type of Needs Assessment 2 .94 2.23 '.TISO 

Error 52 .42 

Technical Assistance ActivitUs Identified 

Type of Needs Assessment 2 .58 .91 .4084 

Error 52 .64 

Roles/Responsibilities Detemiined 
Type of Needs Assessment 2 1.46 3.46\ .0389 

Error 51 .42 

NEEDS ASSESSOR RATINGS ^ 
J 

Comprehensive Review ' > 

Type of Needs. Assessment 1 2/0 2.11 .1557 

Error 33 U28 

Project Needs Identified 
Type of Needs Assessment 1 .48 .50 .4835 

Error 32 .96 

Technical Assistance Needs Identified 
Type of Needs Assessment 1 v07 .02 .8830 

Error 34 .78 ■ 

Technical Assistance Activities Identified 

, Type of Needs Assessment 1 .43 .45 .5091 

Error 34 .96 

Roles/Responsibilities Determined 
Type of Needs Assessment 1 .43 .67 .4186 

Error 34 .64 




£<.05 
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Table 20 

Scheffe Multiple Comparison Test: Project Staff Ratings of 
Extent to Which Roles/Responsi bill ties Were Determined During 
the Needs Assessment for On-Slte {vi). Telephone (1^2) and-Self- 
Admlnlstered (u 3) Strategies 













Contrast 

















1 . 


V\ 


- 1^2 


' .58 


.21 


2.82 


2. 


V\ 


- W 3 


.20 

'/ 


.20 


.99 


3. 


V3 


- V2 


. 38 


.21 


1 .83 



/ 



*F(2,50)£ = .05 » 2.76 
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Table 21 

Analysis of Varlartce: Number of Needs Identified 



Source of Variation 




MS F 


E 


Type of Needs Assessment 
Error 


2 
55 


.37 .21 
1.80 


.8150 




Table 22 






Chi-Square: 


Number of Needs Which Cnanged 




Area of Comparison 


df 


2 

X E 




Needs that Changed 


' 2 ■ 


1 .861 <.50 
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Table 23 



Ch1-Square: Additional Accomplishments of the Needs Assessment 



Respondent 



df 



Project Staff 
Needs Assessors 



1 



1.098 
.286 



.5775 
.5926 



Table 24 , 

Analysis of Variance: Project Staff Satisfaction with Needs Assessment 



Source of Variation 



df 



MS 



Expectations Met 
Type of Needs Assessrtient 
Error 

Usefulness 
Type of Needs Assessment 
Error 

Qual i ty 
Type of Needs Assessment 
Error 

Overall Satisfaction 
Type of Needs Assessmeht 
Error 



2 

52 



2 
53 



2 

51 



2 
51 



.44 
.52 



1.63 
.59 



1.94 
.61 



1.41 
.71 



.85 
2.76 
3.17 
2.00 



.4320 



,0726 



.0503 



,1458 
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Table 25 

Chi -Square: Persons Indicating that the Strategy 
in Wh,ich They Participated Was Appropriate 



Respondent ■ 


df 


2 

X 




Project Staff 








OS vs T vs S^ 
OS vs T 
T vs S 

S vs OS r 


2 


8.94 
8.91 
1.99 
3.02 


.Oil?; 

.0028 
.1582 
.0822 


Needs Assessors 








OS vs T 

t 


1 


12.94 


.0003* 



^OS = on-site; T = telephone, S = self 
*£<.05 



Table 26 

Chi -Square: Strengths of Strategies 



ERIC 



Respondent 


df 


2 '/ 

X 




Project Staff 
Needs Assessors 


. ; 2 
1 


.798' 
1.178 - 


. ■ .6711 
.2777 




. . '! 

\ 


9 

5.0 
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Table 27 



Chi -Square: \ 

■ . ■ * 


eaknesses ,of Strategies 


R,espondent 


df ^ 


rjiOjeut OtaTT 

OS vs T vs , 
OS vs T 
' T vs S 
S vs OS 

Needs Assessors 
OS vs T , 


2 6.69 
]' 6.62 
1 2.44 
, 1 1.39 

1 2.06 


.0353* . 
.0101 
.1183 
.2391 

J517 


^OS = on-site; T = telephone; 
£<.05 


S = self - 





• 

■ - , ) 

Kruskal-Wallis: Calendar Daj 


Table 28 

T 

^s Taken to Conduct the Needs Ai 

: . _ 


>sessments 


c ' — ^ 

Area of^tiniparison 


df H 


S. 


Calendar Time , 


2 212.35 

♦ 


<.001* 


*P<.05 ^ , 
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Table 29 



Median Tests: Comparison of the Strategies by the Calendar Time Taken to 
, Complete the Needs Assessment 



Comparison 


df 


2 

X 

>. 


* 


OS vs T* 


1 


17.26 


<.oor 


T vs S 


1 


6.26 


<.02* 


OS vs S 


1 


7.47 


<.01 



*0S = on-site, T= telephone, S = self 
*£<.05 
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Table 30 

Analysis of Variance: Person Hours 
to Conduct the Needs Assessment 



— ■- ^ 

Source of Variation 


df' 


MS 


F 


. J3L 


Type of Needs Assessment 
Error 


2 

55 


2116.34 
586.46 


3.61 


.0337* 



£<.05 
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Table 31 

Scheffe Multiple Comparison Test: Aniount of Time 
Takfen to Conduct an On-Site (vi). Telephone {^2), 
and Self-Administered {^2) Assessment 



Contrast 




0^ 












1. Ul - P2 


21.48 


7. '566 


2.8391* 


2. 


9.91 


7.309 


1.3557 


3. ^3-1^2 


11.57 


7.405 


1.5624 


*F(3.60)p = 


.05. = 2.76 . 







So 



S3 



Table 32 

Analysis of Variance: Person Hours 
for Follow-up Agreement Development 



Source of Variation df^ ^ L 




Tvoe of Needs Assessment 2 6.60 1.02 
Error 54 o.4t) 


.3663 


Table 33 




Analysis of Variance: batisTaction witn 
' -Individual Tec-hnlcal Assistance Services 

r , 




Source of Variation df MS F 


£. 


Type of Needs Assessment 2 .9/ i -oo 
Error 1^7 .52 




TaKI o 
1 0 D 1 6 «3H 




Analysis of Variance: Progress In Meeting Needs 




Source of Variation df. MS F 


E 


Type of Needs Assessment 2 1.51 1.51 
Error 123 1.01 


.2259 



54 ' ' . ■ %. 



Table 35 

Analysis of Variance: Impacts of 
All, Technical Assistance Services i 



Source of Variation ' df MS F . ' £ 



343.03 1.24 .2999 
276.51 

216.35 .75 ' .4807 

290.15, 



Table 36 

Analysis of Variance: Overall Satisfaction 
With All Technical Assistance Services 



Source of Variation 


df 


MS 


F 


E 


Tyoe of Needs Assessment 


2 


.08 


.14 


.8667 


^rror 


42 


.53 







i 



i 



Organizational Impacts 

Type of Needs Assessment 2 
Error k 41 



Programmatic impacts 

Type af Needs Assessment 2 
— . Error 41 
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Appendix 6 



ADDITIONAL FINDINGS 



INTRODUCTION 

As the. study was being conducted, additional questions and findings 
emerged. They were not as directly related to the overall question of the 
comparability of the three strategies as those presented in the text, but 
shed some additional light on aspects of the assessments. Areas In which 
additional findings were, explored included: (a) project staff satisfaction; 
(b) project staff perceptions of their needs; and (c) Impacts of technical 
assistance which were particular targets or goals of the needs assessments. 
A description and discussion of these findings are given here. 

PROJECT STAFF SATISFACTION ' 

Client's perceptions of satisfaction are an important factor in the 
success of technical assistance (Lillie and Black, 1976). TADS' previous 
evaluation data on providing technical assistance indicate that satisfaction 
varies by year of project operation and that it changes over time (Suarez 
& Vandiviere, 1978; 1979; 1980). For these reasons additional analyses 
were conducted of project staff satisfaction with:^ (a) the needs assessment; 
(b) the individual technical assistance services provided as a result of 
the needs assessment; and (c) all of the technical assistance provided 
during the year, i.e., workshoos, the needs assessments, individual TA 
and publications. Analyses were conducted to determlne^^^^Ratisfaction 
ratings differed by type of needs assessment in which tn^project staff 



\ 



ERIC 



57 

i 

participated, year of project funding and/or If there was an Interaction 
between the two. For satisfaction with the needs assessments and overall 
satisfaction, there were small' npmUers of projects represented In some of 
the categories. These results should therefore be considered tentative. 

SatUiactA-on Mith tha Hzo-db k&huhmznt. As mentioned In the body of 
this report, satisfaction with the needs assessment was defined as project 
staff ratings of four items: (a) the extent to which exoectations were 
met by the needs assessment; (b) the usefulness of the needs assessment, 
(c) its quality, and (d) overall satisfaction. Previous analyses conducted 
at TADS of these types of data suggested that the items might be highly 
related to one another. For this reason a principal components factor 
analysis was cpnducted to see if the items were measures of the saii)e 
characteristic or factor. The results indicated that the items did indeed 
represent one factor--one which was given the overall title of "client 
satisfaction" (see Statistical Table 39). 

Using the results of the factor analysis, the responses from each 
project to the four items were converted to a one factor score representing 
client satisfaction with the needs assessments. (It is because of this 
analysis that some of the resuTting means were negative numbers, e.g. , 
Figure 4.) To determine if there were differences in client satisfaction 
among the needs assessment strategies or among years of funding, an unbalanced 
two-factor analysis of variance was conducted using the^ factor scores as 
the dependent variable. Results of the analysis revealed a significant 
Interaction between type of needs assessment and year of funding (see- 
Statistical Table 40). 
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As Figure 1 Illustrates, there were noticeable differences In satis- 
faction ratings among the staffs In different years of funding who oarticl- 
pated In on-site and telephone needs assessments. First year pj-oject staffs 
were >ess satisfied with on-site needs assessments than were second er third 
year projects. First and third year project staffs were considerably more 
satisfied with telephone needs assessments than were second y^ar projects. 
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Figure 4 

Mean Factor Scores of Project Staff Satisfaction 
with the Needs Assessments by Type of Needs Assessment 
and Year of Funding 



u 
o 
u 
m 

o 
u 



+2.0 

ti.o 

0.0 

I 

rl.O 

-2.0 
-3.0 
-4.0^ 



-5.0 



First Hear Projects 
Second Year Projects 
Third Year Projects 




I I I 

On-Site Telephone Self 



ScutUicicXlon uujth IndUvlduaZ Tachnlcal Ai,6l6tance. SeAv^cii. Satisfaction 
with TA services was reported by the clients after each service by rating the 
item, "your overall satisfaction with the service," on a six point scale 
(1 ■ unsatisfactory and 6 • exceptional). Clients oarticipating in the on-site 
and telephone needs assessment tended to rate their satisfaction with individual 
services (M = 5.23 and M = 5.22, respectively) slightly higher than those 
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clients participating in the self -administered needs assessment (M = 4.98). 
As was reported in the text, these differences were not statistically signif- 
icant. Addittonal analyses indicated there to be a statistically significant 
difference in satisfaction with TA services for projects in different years 
of funding, but the interaction between tyoe of needs assessment and year of 
funding was not statistically significant (see Statistical Table 41). 
Satisfaction with TA services increased as the year of funding increased 
(i;e., first year projects, M = 4.88; second year projects, M = 5.02; and 
thrid year projects, M = 5.49). These differences in satisfaction with 
individual TA services are consistent with the findings in previous years 
(Suarez A Vandiviere, 1978; 1979; 1980). 

OvzAdU ScutUiacUon uxUh Technical AAAl&tance.. Satisfaction with 
all technical assistance was reported by the projects at the end of the year 
by rating the item, "your satisfaction with this years TADS technical assistance, 
on a seven point scale (-3 to +3; -3 = extremely dissatisfied and -♦■3 = extremely 
satisfied). The trend for projects' ratings of satisfaction at the end 
of the year with all technical assistance was similar to the findings for 
satisfaction with individual TA services, (i.e., projects participating in 
on-sjte needs assessments, M » 2.50; telephone, M = 2.43; and self- 
administered, M « 2.41). The trend by year of funding was also similar 
,(1.e., first year, M = 2.37; second year, M => 2.47; and third year, M = 2.55). 
Even though both of these trends were similar to the finding for satisfaction 
with Individual TA services, additional analyses indicated there were NO 
statistically significant differences by type of needs or year of funding, 
and there was NO significant interaction between year of fundi na and type 
of needs assessment (see Statistical Table 42). 



PROJECT STAFF PERCEPTIONS OF THEIR NEEDS 

Data was gathered to Investigate one aspect of project staff perceptio 
of their needs. That one aspect was the project staff's ratings of the 
status of the identified needs before and after receiving' individual ized 
assistance with each neid. As reported in the text, the differences in 
these status ratings also provided a measure of "progress in meeting needs" 
of the projects. Additional data were collected and additional analyses 
conducted that were not reported in the text. The results of these 
additional analyses provided some insight into the pr*ojects pierceptions 
of their needs. 

The status of the identified needs, before technical assistance, was 
collected twice during the year. The first time was immediately after 
the needs assessments and before the individualized assistance to address 
the need was provided (this was the additional data not reported in the 
text). This status rating was labeled "Time 1 Initial Status" (see Table 
37). The second time was a retrospective rating of the initial status 
which was collected at the end of the year. This second status rating was 
labeled "Time 2 Initial Status." The status of the identified needs 
after receiving individualized assistance was gathered once, at the end of 
the year. This status rating was labeled "Final Status." 

There was considerable variations between the initial status at 
"Time 1" and "Time^ 2" (see Table 37). The project staff's perception of 
the 1r»1t1al status at "Time 2" was higher than at "Time 1" for all groups, 
except those groups of projects who were in their first year of funding 
and participated in the self-administered needs assessment. In fact,' even 
when those first year projects who participated in the self-administered 
needs assessments were included, the difference in the overall means for 
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the status ratings at "Time 1" and "Time 2" was highly statlstj^cally 
significant ("Time'l," M » 1.79; "Time 2," M = 2.22; correlated t = 3, 
df - 147; £ - .001). , 
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Of particular Interest were the ratings pf first year project staffs. 
The areatest difference In Initial status ratlt^^s was given by^ first year 
project staffs who participated In an on-site needs, assessment ("Time 1" - 
"Time 2" - .96). The?e projects rated their initial status ("Time 1") 
lower than any other group. At the end of the year they perceived their 
initial status ("Time 2") much higher and comparable with the ratings 
other first' and second year project staffs. For those first year staffs 
who participated in a self-administered assessment, a very different 
phenomenon occurred. They rated their initial status after the needs 
assessment T'Time 1") higher than at the end of the year ("Time 2")— the 
only group to do so. While the difference between "Time 1" and "Time 2" 
ratings (.86) of third year staffs participating -in self assessments 
was not as high as first year staff In on-site assessments, it was a 
large differience. In additi'on, both initial status ratings were higher 
for this group than for any of the others. 

Additional analyses were conducted to investigate differences between 
initial status and final s\^us by year of funding, and to investigate 
interaction effects between year of funding and type of needs assessment 
(this was labeled "progress in meeting needs" in the text). The "Time 2*'-,^ 
status ratings were used as the initialstatus durirtg these analyses 
(also In the text) because these ratings were most often higher than 
"Time 1" ratings and, therefore, provijle^ a more conservative view of 
the differences between initial and final status. (Also, initial status 
of needs In previous years was only collected at "Time 2.") While the 
differences between initial and final status by year of funding were 
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'Wo\ statistically significant, the tnteraction between the type of needs 

aS?essme*)t and yeair of project funding was statistically significant 

(«ee Statistical Table 43). Figu^^e^i? provides a qraphic representation 

of this interaction effect. As may be ndted, third year project's who 

parti<?lpated in the on-site assessment reported the llirgest change 

between initial and final status. The smaltest extent, of change was 

for third year projects who oarticioated in the self -administered assessment. 

Result! for first year projects participating in on-site and self -administered 

assessm^ts were the reverse of those found for third year projects. These , 

findings. While interesting, appear to be a result of. the differences in 

ratings given\o the initial status ("Time 2") rather than the final status. 

• Differences among average ratings^ of change in status for atl second year' 

projects and all projects participating in the tele^phone strategy were not . 

Figure 5 . - 

MEAN CHANGE by Tyoe of Needs Assessment 

and Year of Funding . 
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TARGETED IMPACTS OF TECHNICAL ASSISTANCE 



be 



Needs assessments, in and of themselves, are considered by TADS to 



r 



an' important technical assistance service to clients. Because it 



impacts of tSqhriical assistarjice 
or09nizational impacts assessed 
by 



involves a total program review^ the assessment is believed to be a 
sei^vice which aids in the future organization and operation^of^^th^^ 
Drc^ject. The process should, tfjierefore, result in certain organizational 

on the„ projects. In relation to the » 
at the end of the year, it was determiniBd 
TADS th^it the needs assessmeri^ should produce impacts in the spepific 
ardas of: (a) decision-making; (b) program analysis; (c) staff role 
,an4 responsibility definition;™ sjnd (d) program clarification. ,An ^ 
adqitional s6t of analyses was conducted to determine^whethers or not 

which there were any differences in 

ch clients oartici- 



0 




th'^s occt^rreJl and the extent to 
these ^areas when the.,^pe of neejds assessment 
paled was considered. , | ^ j 

To obtain information reqarjding the extent of the targeted impacts, a^ 
coiint of the positive impact railings (there were no negative impacts indicated 
by any project) was> done (see Ta|ble 38). A large, majority of" the projects 
. reported positive impacts tn three* of the four target areas, and the 
nurT|ber of impacts were approximately equal for orojects participating 
in ithe different needs assessment strategies. The one target area 



which did not have as many reported positive impacts was "staff roles." 
This item was described as "identifying and/br clarifying^aff roles 
and responsibilities." Less than half 6f the projects participating 
in the self-administered assessments reported positive impacts. Slightly 
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Table 38 



FrequencleSi Means and Standard Deviations 
of Project Staff Ragins of Targeted Organizational Impacts 







Type of Needs Assessment 


Area of Comparison 




On-Site 
(n=15) 


Teleohone 
(n=17) 


Self 
(n=17) 


NUMBER OF PROJECTS INDICATING 
AN IMPACT 










Decision Making 




14' 


14 


15 


Program Analysis 




14 


17 


15 


Staff Roles 




12 


9 


5 


Program Clarification 




13 


16 


15 


LEVEL OF IMPACT RATINGS 










Decision-Making 


M 

SD 


2.33^ 
.90 


1.71 

U05 


1.5S 
.94 


Program Analysis 


M 
SO 


2.33 

.82 


2.41 
.77 


2.00 
J. 06 


^ Staff Roles- 


M 

SD 


1.47 
1. 13 


.88 
1.05 


.65 
7. II 


Program Clarification 


M 


2.07 
MO 


2.00 
.87 


1.71 
.99 



^Based on a 7 point scale (-3 to +3) where -3 = significant negative 
impact, -2 = moderate negative imoact, -1 = limited negative impact, 
0 = no impact, +\ = limited positive impact, +2 = moderate positive 
impact, and +3 = sianificant positive impact. No negative ratings - 
were given, by respondents. The means, therefore, reflect averages 
on a 4 point scale, i.e., 0-3. 



more than half the projects participating in the on-site and telephone 
assessments reported positive imoacts in the "staff to^es" area. 

Means and standard deviation were computed to determine the over- 
all level or intensity of the targeted impacts. The general pattern of 
the means in Table 38 indicates that projects participating in the on-site 
assessments reported the Mghest level of impact followed by the telephon- 
assessment, and then self-administered assessments. An exception to this 
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pattern may be noted in the "oroaram analysis" area. In this area, the 
highest level of impact was reoorted by projects oarticipating in the ' 
telephone assessrapnt. To determine if these differences were statistically 
significant, one-way analyses of variance were computed for each target 
area. No statistically significant differences were found (see Statistical 
Table 44). 

SUMMARY 

It has been the purpose of this discussion to provide additional 
information related to the needs assessments. In general, ft appeared 
that the results described three tentative and initial findings. First, 
^jj^ient satisfaction with technical assistance apoears to change over time. 
The most specific differences in client satisfaction and among project 
staffs in different years of funding were observed prior to the provision 
of technical assistance to meet identified needs. Differences in client 
satisfaction with subsequent TA shii'ted during the year as services were 
provided and were found to be only among projects in different years of * 
funding. At the end of the year, differences in client! satisfaction with 
TA were no longer evident. 

Secondly, project staff pe^eptions of their needs, in relation to 
their status, also apoears to change over time. Retrospective perceptions 
of initial status ("Time 2") tended to be rated higher than the initial 
status ('*Time 1") reported immediately after needs assessments. 

Finally, in three of four areas targeted specifically for the needs 
assessment, desired impacts were reported by most of the projects. Impacts 
relative to staff role clarification varied in the same manner as the 
extent of involvement of a trained consultant during the needs assessment, 

ERJC 7^ 
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I.e., the greatest Impact in this area was for projects participating In 
assessments which had higher needs assessor Involvment, the on-site assessment. 
This result;- cannot be entirely attributed to the type of needs assessment 
because these Impact ratings were reported at the end of the year after 
all technical assistance services. However, this result did support other 
findings in the text which describe the benefits to project staffs from 
direct, personal assistance. 
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Table 39 



Factor Analysis: Satisfaction Items 



Items 



Weight 
on Factor 



Expectations Met 
Usefulness 
Quality 
Satisfacltion 



.87 
.87 
.91 
.95 



Eigenvalue 

Percent of Variance Explained 



3.24 
81% 
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Table 40 

Analysis of Variance: Satisfaction with Needs Asses sn«nt 

(Factor Scores) 



Source of Variation 


df 


MS 






Type of Needs Assessment 


2 


21.22 


f 

2.47 


.0944 


Year of Funding 


2 


10.68 


1.25 


.2966 


Interaction 


4 


26.63 


3.10 


.0233* 


Error 


50 


8.58 







*£ <.05 



Table 41 

Analysis of Variance: Satisfaction with Individual 
Technical Assistance Services 



Source of Variation 




MS 


F 




Type of Needs Assessment 


2 


.97 


1.85 


.1608 


.Year of Jjiading 




5.58 


10.66,., 


•QQOl* 


Interaction 


4 


. .37 


.76 


.5524 


Error 


147 


.52 







*£ <.05 



Table 42 

Analysis of Variance: Satisfaction with all 
Technical Assistance Services 



Source, of Variation 


, df 


MS 


F 


' £ 


Type of Needs Assessment 


2 


.08 


.14 • 


.8667 


Year of Funding 


2 


.18 


.34 


.7150 


Interaction 


4 


.45 


.84 


.5073 


Error 


. 42 


.53 
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Table 43 

• Analysis of Variance: Progress In Meeting Needs 



Source of Variation ' df MS F p 



Type of Needs Assessment 


2 


1.52 


1.51 


.2259 


Year of Funding 


2 


.02 


.02 


.9800 


Interaction 


4 


2.62 


2.79 


.0292* 


Error 


123 


-1.01 







♦£^..05 



Table 44 

Analysis of Variance: Targeted Organizational Impacts 

Source of Variation 1£ £ £ 

\ 

Decision Making 

Type of Needs Assessment,' 2 2.51 2.69 .0788 

Error 46 .93 

Program Analysis 

Type of Needs Assessment 2 .80 1.04 .3600 

Error 46 .77 

Staff Role Clarification 

Type of Needs Assessment 2 2.79 2.33 .1091 

Error 46 1.20 ' 

Program Clarification 

Type of rfeeds Assessment 2 .60 .63 .5391 

Error 46 .97 



7. 



